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ABSTRACT 

Sustainability education and conservation have become an international imperative due to the rising cost of energy, 
increasing scarcity of natural resource and irresponsible environmental practices. This paper presents Makahiki, an open 
source serious game framework for sustainability, which implements an extensible framework for different organizations 
to develop sustainability games. It provides a variety of built-in games and content focused on sustainability; game 
mechanics such as leaderboards, points, and badges; a variety of common services such as authentication, real-time game 
analytics and ability to deploy to the cloud, as well as a responsive user interface for both computer and mobile devices. 
The successful implementation of six sustainability educational games in different organizations provides evidence 
regarding the ability to customize the Makahiki framework successfully to different environments. 
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1. INTRODUCTION 

The rising cost, increasing scarcity, and environmental impact of fossil fuels as an energy source makes a 
transition to cleaner, renewable energy sources an international imperative. Moving away from petroleum is a 
technological, political, and social paradigm shift, requiring citizens to think differently about energy 
policies, methods of generation, and their own consumption than they have in the past. Unfortunately, unlike 
other civic and community issues, energy has been almost completely absent from the educational system. To 
give a sense for this invisibility, public schools in the United States generally teach about the structure and 
importance of our political system (via classes like “social studies”), nutrition and health (through “health”), 
and even sports (through “physical education”). But there is no tradition of teaching “energy” as a core 
subject area for an educated citizen, even though energy appears to be one of the most important emergent 
issues of the 21st century. 

Another emergent issue is the explosive spread of game techniques, not only in its traditional form of 
entertainment, but across the entire cultural spectrum. The adoption of game techniques to non -traditional 
areas such as finance, sales, and education has become such a phenomenon that the Gartner Group included 
“Gamification” (Deterding 2011) on its Hype List. 

This paper describes Makahiki, an open source serious game framework for sustainability education and 
conservation, in which we attempt to create synergy between these two emergent issues. The result of over 
three years of research and iterative development, Makahiki explores one section of the design space where 
virtual world game mechanics are employed to affect real world energy behaviors. The goal of the Makahiki 
project is to provide a framework for organizations to learn to not just affect energy behaviors during the 
course of the game, but to produce long lasting, sustained change in energy behaviors and outlooks by 
participants. 

We initially used Makahiki to create an energy challenge called the Quest for the Kukui Cup (hereafter, 
the Kukui Cup) for approximately 1,000 first year students living in four residence hall towers at the 
University of Hawaii in 2011. During the three weeks of the competition, over 400 of the eligible students 
played the game, for a total of 850 game play hours. The game mechanics were designed to create a 
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self-reinforcing virtuous circle between the real world and virtual world activities. The challenge was well 
received and has been repeated in academic years 2012 and 2013 at the University of Hawaii. In addition, 
Makahiki, as a serious game framework, has been used in several organizations including Hawaii Pacific 
University and the East West Center, an international education institute in Hawaii, to implement their own 
version of the Kukui Cup serious game. 

In this paper, we detail the innovative features of Makahiki framework, along with our experiences using 
them to implement several serious games related to sustainability education and conservation in different 
organizations. 


2. RELATED WORK 

Our research draws on previous work done in the areas of energy behavior research, energy competitions, 
gamification, and serious games. To reduce energy consumption, providing energy feedback is a critical 
foundation. Darby's survey of energy consumption studies from the past three decades found that 
consumption in identical homes could differ in energy use by a factor of two or more depending on the 
behavior of the inhabitants (Darby 2006). Another survey of energy feedback conducted by Faruqui et al. 
found that residents that actively used the in-home displays with near-real-time feedback averaged a 7% 
reduction in energy usage (Faruqui 2009). Darby also points out that feedback alone is not always enough: 
other factors such as training could lead to higher rates of energy conservation (Darby 2000). 

Energy competitions or challenges have been introduced to college dormitories and residential homes as 
ways to facilitate and incentivize energy reduction. Petersen et al. describe their experiences deploying a real- 
time feedback system in an Oberlin College dorm energy competition in 2005 that includes 22 dormitories 
over a 2-week period (Petersen 2007). Web pages were used to provide feedback to students. They found a 
32% reduction in electricity use across all dormitories. The Building Dashboard (Lucid Design Group 2008), 
developed by Lucid Design Group, is used to support Oberlin's dorm energy competition, as well as the 
Campus Conservation Nationals, a nationwide electricity and water use reduction competition on college 
campuses (Lucid Design Group 2011). The Building Dashboard enables viewing, comparing and sharing 
building energy and water use information on the web in compelling visual interface, but the cost of the 
system creates the barrier for wider adoptions. In addition, the building dashboard solutions focus on 
providing energy information as a passive media. There is little interaction between participants and the 
system. 

Games on the other hand, have been shown with great potential as successful interactive media that 
provide engaging interfaces in various serious contexts (McGonigal 2011, Reeves 2009). Priebatsch attempts 
to build a game layer on top of the world with his location-based service startup (Priebatsch 2010). 

Reeves et al. described the design of Power House, an energy game that connects home smart meters to 
an online multiple player game with the goal to improve home energy behavior (Reeves 2011). In the game, 
the real world energy data are transformed into a “more palatable and relevant form of feedback”, and players 
may be incentivized by the in-game rewards to complete more energy- friendly real-world behaviors. 

ROI Research and Recyclebank launched the Green Your Home Challenge as a case study of employing 
gamification techniques online to encourage residential green behavioral changes offline (Haiges 2011). 
Working with Google Analytics, the results show a 71% increase in unique visitors and 97% of participants 
surveyed said that the challenge increased their knowledge about how to help the environment. 

The blending of real and virtual worlds has been explored in broader contexts. McGonigal designed the 
award winning serious Alternative Reality Game (ARG) “World Without Oil” (Electric Shadows 2010) and 
later “Evoke” (World Bank 2010) with the goal to empower people to come up with creative solutions to our 
most urgent real-world problems. ARGs have also been used to support learning. Connolly et al. discuss the 
development of an educational ARG to motivate secondary school students across Europe to learn foreign 
languages (Connolly 2009). The results of the pilot run of the game in 2009 indicated that 92% of students 
felt the game motivated students to learn a second language. One of problems the team identified is the 
limitation of Moodle platform the game is based on. 

The report of the ARGOSI project provides insights to the use of ARGs in game based learning and the 
challenges in the field of higher education (Whitton 2009). The pilot was run at the University of Bolton with 
the aim to provide an engaging alternative to traditional methods of introducing students to university life. 
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The overall up-take of the game was fairly low with only 23 active players out of 173 total. The project 
identifies a number of questions surrounding educational ARGs, such as motivation, relationship to 
curriculum, marketing and timing. The report suggests that a complete ARG model may not be appropriate 
for wholesale learning, but there is certainly potential in using game elements. 


3. MAKAHIKI SYSTEM DESIGN 


Makahiki is an open source “serious game framework for sustainability”. It provides a framework for 
creating serious games for the purpose of education and behavioral change regarding energy, water, food, and 
waste generation and use. Makahiki intends to create synergy between the need to create knowledge and 
engagement regarding energy and the ability of so-called “serious game” techniques and energy feedback to 
create participation and engagement. 

Makahiki consists of a configurable game engine that can be customized to the needs of different 
organizations. It includes a library of pre-built game “widgets” that implement a variety of game mechanics. 
Using the widgets, an organization can create a custom energy challenge in which players can compete 
individually and/or in teams to earn the most points by reducing their energy consumption as well as by 
learning about energy concepts in general. 

Figure 1 illustrates a home page of the system implemented by using the Makahiki framework. 
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Figure 1. Makahiki home page 


3.1 Architecture 

Figure 2 illustrates the overall architecture of Makahiki. 
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Figure 2. Architecture of Makahiki 


133 



ISBN: 978-989-8533-30-2 © 2014 


The core component of Makahiki is a configurable game engine that can be customized to the needs of 
different organizations. It includes two libraries of games and game mechanics. These libraries consist of a 
set of pre-built “widgets”. By selecting and configuring these game and game mechanics widgets, an 
organization can create a customized serious game. 

Makahiki interfaces with the outside environment in three different ways. First, the top side of the 
architecture diagram shows that Makahiki has two primary user interfaces: one for the players of the serious 
game, who directly interact with the game and game mechanics widgets; the other for the administrators of 
the system, who configure the system and monitor the real-time game analytics. 

Second, the right side of the diagram illustrates that Makahiki must obtain real-world environmental data 
as the challenge progresses in order to provide feedback to users about the impact of their actions. In some 
cases, environmental data can be input automatically into the system through a combination of “smart” 
meters and additional services, such as WattDepot (Brewer 2011) for energy data collection, storage, and 
analysis). If that is not possible, then manual meters can be read by administrators on a regular (typically 
daily) basis and input into Makahiki using and administrator interface. 

Third, the bottom side illustrates that Makahiki stores its data in a database repository (currently 
PostgreSQL). To reduce database access and improve performance, Makahiki provides support for caching 
(currently memcached). 


3.2 A Library of Configurable Games and Mechanics 


Makahiki builds in a set of configurable games and mechanics that can be turned on or off, or customized by 
the game designers to the needs of different organizations. 


3.2.1 Energy and Water Game 


A fundamental requirement for enabling more active participation in sustainability behavior is feedback 
regarding their resource such as energy and water usage. The Energy and Water game in Makahiki are 
implemented as the Daily Resource Goal Game. The Daily Energy Goal Game widget provides a way for 
players to see the outcome of the energy reduction behavior, and to make it a game by earning points from 
their behavior. By reducing their teams' daily energy consumption from a baseline by a set percentage, the 
players in the team will all earn the configured amount of points. Figure 3 illustrates this widget. 
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Figure 3. Daily Energy Goal Game Widget 


This interface uses a stoplight metaphor to show at a glance whether or not the team is making the goal. 
In this case, the stoplight is green, indicating they are currently below the goal. We have found additional 
perspectives of the energy feedback to also be useful. One useful perspective to a team is a real time power 
meter visualization that shows the current power usage of a team. This visualization displays the real time 
power consumption which updates at a specified interval such as 10 seconds. Another useful perspective to a 
team is a historical, calendar-based visualization that shows the results of the energy goal game for each day 
of the current round. 
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3.2.2 Smart Grid Game 

Smart Grid Game (SGG) is Makahiki's approach to support “gamified” delivery of educational experiences. 
Educational actions are organized into a grid of squares (hence the name “Smart Grid”) and organized by 
category columns and levels. Players use its grid interface to discover “actions” they can perform. Successful 
completion of an action earns the player a variable number of points depending upon the difficulty of the 
action, and can potentially “unlock” additional actions and higher levels in the SGG. Figure 4 shows a typical 
Smart Grid Game interface for players: 
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Figure 4. Smart Grid Game widget 


To make your SGG more interesting to players, and more pedagogically sophisticated, Makahiki supports 
the definition of “path” through the educational content or actions. In most cases, when a new player sees the 
SGG for the first time, there should only be a few actions available to them, possibly only one. All of the rest 
should be locked. Makahiki provide a set of predicates that can be used to define the path. The predicates 
determine if an action or level is locked or unlocked for a player, which in term depends on the outcome of 
another action or multiple other actions. 

3.2.3 Raffle Game 

The Raffle Game widget provides a way to incentivize participation from all individuals, even those who are 
not in the running for a top prize. For every 25 points a player earns, they receive one virtual raffle ticket. 
Players can dynamically allocate their tickets to any raffle prizes they are interested in at any time, up to the 
end of the raffle. Figure 5 shows an example of the Raffle Game. 



Figure 5. Raffle Game widget 

Raffle tickets are independent from a player's score, and allocating a raffle ticket does not affect their 
rank. The system provides random selection of the winner of each raffle item at the end of a round. 

3.2.4 Social and Referral Bonuses Game Mechanics 

The Social and Referral Bonus widgets are the game mechanics that help encourage participation by 
providing additional points to players who participate in activities with other players, and facilitate the entry 
of new players into an energy challenge. The social bonus is a configurable option when an action is created 
in the Smart Grid Game. Players earn extra points if they perform the action with another player. When a 
player submits a response for an action with a social bonus, the player can provide the email address of the 
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person who jointly completed the action. Once the other player completes the action, the social bonus is 
awarded. 

Players are led through a setup process when logging into Makahiki for the first time. One of the steps in 
this process is the referral bonus. If a player was referred by another player in the system, he can use this step 
to input their email address. Once the new player earns a certain number of points in the competition, both 
players are awarded a referral bonus of a configurable number of points. 

3.3 Real-time Analytics 

Makahiki is designed to support energy challenges involving hundreds or thousands of users lasting weeks or 
months. In these circumstances, effective use of the technology requires the ability to understand the state of 
the game, such as: Who is using it? What are they doing? What is the player response to activities, 
commitments, excursions, and events? Such state information is important for planning purposes, such as 
assessing the transportation needs for an upcoming excursion by seeing how many players signed up. It can 
also be used for making in-game changes to game design, such as changing the point values associated with 
activities to encourage or discourage participation. It can also help identify breakdowns in game play, such as 
significant numbers of unallocated raffle tickets indicating that users do not understand the nature of that 
game mechanic. To address these needs and others, Makahiki includes a variety of widgets that work 
together to provide high level overview of game play state to the administrators of a challenge. 

3.4 Cloud Deployment Support 

Another Makahiki feature is the ability to deploy to a cloud platform such as Heroku. Cloud computing has 
the advantage of simplifying IT administration by eliminating the need to acquire and maintain hardware and 
operating system software. This also can dramatically decrease the cost of deployment. Figure 6 shows a 
screen shot of the Dashboard showing the 2012 East West center Kukui challenge deployed in Heroku, one 
cloud platform provider, and the monthly cost for this deployment. 



Figure 6. Heroku cloud deployment 


4. EXPERIENCES WITH MAKAHIKI 

We have used Makahiki to create six different energy and water serious game instances, all called “Kukui 
Cup” challenges. Three Kukui Cup Energy challenges were held at the University of Hawaii (UH) in 2011, 
2012 and 2014 for over 1,000 first year students each year living in the residence halls. Hawaii Pacific 
University (HPU) held a Kukui Cup Energy challenge in 2012 and 2013 for about 200 students each year. An 
international organization called the East-West Center (EWC) held a Kukui Cup Energy and Water challenge 
for approximately 600 international residents living in their residence halls in 2012. Since the EWC residence 
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halls did not have internet-enabled meters, resource consumption data had to be entered by the game 
managers manually. 

The successful creation of these serious game challenges by three different organizations provides 
evidence that Makahiki can be successfully tailored to the needs of different organizations. First, UH and 
HPU used different metering infrastructure, and EWC collected their resource data manually. Second, while 
UH and HPU challenges involved only energy consumption data, the EWC challenge involved both energy 
and water consumption data. Third, the IT infrastructure at UH and HPU provided authentication services 
using CAS (Central Authentication Service) and LDAP, while EWC used the built-in Django authentication. 
Fourth, the user interface was customized to “brand” each challenge with the logo, thematic elements, and 
the education contents of the sponsoring organizations. 

The evaluation of Makahiki includes both qualitative and quantitative sources of data regarding the 
system. Makahiki provides custom quantitative instrumentation that enables us to track when, where, and for 
how long each player accessed each page of the site. Unlike generic web server logs, we could track per- 
player application-specific behaviors. We also gathered qualitative data through a survey that players could 
complete as part of a Smart Grid Game activity during the final week of the competition. The survey asked 
participants to provide short answers to questions regarding the way the competition and website was 
designed. 41 players completed this survey. 

In response to a survey question asking how the player might describe the Kukui Cup, 83% said “Fun”, 
95% said “Educational”, while 7% said “Difficult” and 2.3% said “Boring”. In response to the question, 
“What was confusing in the website”, 46% of the players said “Nothing”, and 32% of the players also 
responded “Nothing” in response to the question, “What would you change about the website? When asked 
what they liked most about the website, 60% of the survey respondents said “ease of use”. Instrumentation 
also indicates that the game was generally easy to use. 73% of the 418 players never accessed the “Help” 
page, and only 5% of the players sent questions to the administrators. The data bear out the success of the 
raffle game. Players mentioned the raffle game repeatedly as the most interesting incentive in the game, and 
over half the students with at least 100 points participated in the Raffle. 

To assess the effectiveness of the framework for designing games that improve player literacy in 
sustainability, we conducted two energy literacy surveys during the 2011 Kukui Cup Challenge at the 
University of Hawaii at Manoa. One survey was administrated before the challenge (pre-game) and one after 
(post-game). 24 players completed both surveys. Out of the total 19 energy literacy questions, the average 
number of questions answered correctly is 7.54 before the challenge, and 8.96 after the challenge. This result 
indicates an 18% improvement on the energy literacy. We also surveyed non -players as a control condition, 
and found that their literacy did not change, indicating that the improvement in player literacy was indeed 
due to the game. 

To assess the effectiveness of the framework for designing games that produce positive change in 
sustainability behaviors, we recorded and analyzed energy consumption data before, during and after the 
challenge. Before the challenge, an energy usage baseline was established. During the challenge, compared 
to the baseline, 12 out of the total 20 teams reduced their energy consumption, with the highest reduction of 
16.1%. However, 3 teams actually increased their energy consumption, with the highest increase of 11.7%. 
Overall, the average reduction of the 20 teams was low, approximately 2%. 

We also assessed player engagement of the game. We calculated a variety of engagement metrics based 
on the analytics data collected by the Makahiki framework. The participation rate of this challenge is 37%, 
which is good compared to other sustainability challenges. Over the course of the challenge, an average 
player spent about 27.7 minutes per day on the website. One player spent 8.5 hours on one day. There were 
an average of 266 activity submissions and 208 social interactions between players per day. The average 
number of website errors per day was 0.6. The data indicates that Makahiki can be successful in achieving 
player engagement and literacy improvement, although the evidence of positive change in behavior is not 
significant. 


5. CONCLUSION 

The Makahiki research presents an innovative information technology infrastructure that can support 
effective and efficient development of serious games for sustainability education and conservation that can be 
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used by different organizations. Its tailorability and game analytics also provides a useful platform for 
research on gamification, sustainability education, and behavior change. 

Our research suggests that several enhancements to the Makahiki framework would be useful. One is 
real-time player awareness. It is not possible currently in Makahiki to know who is currently “on line” and 
playing the game. Creating this awareness opens up new social gaming opportunities (performing tasks 
together), new opportunities for communication (chat windows), and potentially entirely new games (play 
“against” another online player). 

Makahiki currently ships with over 100 possible “actions” already developed for the Smart Grid Game. 
However, the content is intimately tied to the Smart Grid Game implementation. An enhancement is to 
provide a “content management system” for “actions”, separating “content” from the “presentation”, thus 
new contents can be added and more games can be developed using this content library. 

We are also exploring a future direction involving the development of a consortium of organizations in 
order to scale the use of the Makahiki framework in new settings. Moving outside of the context of either 
Hawaii or college-aged players will necessitate development of significant new forms of content, as well as 
new game mechanics. 


ACKNOWLEDGEMENT 

This research is supported in part by grant IIS-1017126 from the National Science Foundation; the HEI 
Charitable Foundation; Hawaii Electric Company; and the State of Hawaii Department of Business, 
Economic Development, and Tourism. 


REFERENCES 

Brewer, R.S. and Johnson, P.M. 2010. WattDepot: An open source software ecosystem for enterprise -scale energy data 
collection, storage, analysis, and visualization. Proceedings of the First International Conference on Smart Grid 
Communications. 

Connolly, T. et al, 2009. Arguing for multilingual motivation in web 2.0: a games-based learning platform for language 
learning. European Conference on Games Based Learning. 

Darby, S. 2000. Making it obvious: designing feedback into energy consumption. Proceedings of 2nd International 
Conference on Energy Efficiency in Household Appliances and Lighting. 

Darby, S. 2006. The effectiveness of feedback on energy consumption. Technical report, Environmental Change 
Institute, University of Oxford. 

Deterding, S. et al, 2011. From game design elements to gamefulness: Defining “gamification”. Proceedings of 
MindTrek. 

Electric Shadows. 2010. World without oil, http://www.worldwithoutoil.org/metaabout.htm 

Faruqui, A. et al, 2010. The impact of informational feedback on energy consumption-a survey of the experimental 
evidence. Energy , 35(4): 1598-1 608. 

Haiges, H. 2011. Gaming for good. Technical report, ROI Research Inc. 

Lucid Design Group. 2011. Campus conservation nationals, http://www.competetoreduce.org/, Lucid Design Group, Inc. 

Lucid Design Group. 2008. Building Dashboard, http://www.luciddesigngroup.com/, Lucid Design Group, Inc. 

McGonigal, J. 2011. Reality is broken: Why games make us better and how they can change the world. Penguin Press. 

Petersen, J. E. et al, 2007. Dormitory residents reduce electricity consumption when exposed to real-time visual feedback 
and incentives. International Journal of Sustainability in Higher Education , 8(1): 16-33. 

Priebatsch, S. 2010. The game layer on top of the world, http://www.ted.com/talks/seth _priebatsch_the_game_layer_ 

on_top_of_the_ world.html , TED. 

Reeves, B. et al, 2011. Leveraging the engagement of games to change energy behavior. Proceedings of the CHI 2011 
Workshop on Gamification. 

Reeves, B. and Read, J. 2009. Total engagement: using games and virtual worlds to change the way people work and 
businesses compete. Harvard Business School Press. 

Whitton, N. 2009. Alternate reality games for orientation, socialisation and induction (ARGOSI). Technical report, JISC. 

World Bank. 2010. Urgent evoke, http://www.urgentevoke.com/, World Bank Institute. 


138 



